[Spark-2201] Improve FlumeInputDStream's stability and make it scalable #1755
Conversation
|
Can one of the admins verify this patch? |
| * agent.sinks.ls1.router.retry.times=1 [optional] | ||
| * agent.sinks.ls1.router.retry.interval=1000 [optional] | ||
| */ | ||
| public class SparkRpcClient extends AbstractRpcClient implements RpcClient { |
There was a problem hiding this comment.
AbstractRpcClient already implements RpcClient - so you don't really need to implement RpcClient.
|
Can one of the admins verify this patch? |
|
Hey @joyyoj This is a very cool functionality! In fact, we had added a subset of this functionality (sink) as part Spark 1.1, thus validating your approach. However, while this approach provides more functionality, it does at the cost of maintaining this, debugging this, and the cost of addition dependencies like to the spark project, like zookeeper, etc. We are seeing a lot of such big contributions from the community, which is very encouraging and useful, but will be very hard to maintain for us. So we are trying to figure out a way by which the community can contribute such custom functionality, and maintain it themselves. Say something like a contribs repo... I will let you know when we have figured something out. Till then please bear with us. :) |
|
@joyyoj Regarding this PR, mind closing it? |
|
@joyyoj Mind closing this? |
|
Never mind, very very sorry for replay so late. |
5 participants
re-submit